Indirect data searching on the internet

ABSTRACT

The present invention includes an Internet analysis process that includes initializing a data set, accessing a search engine to acquire search results, parsing the search results, rather than a native search engine indexable resource, to output a conclusion, and providing an updated data set. The present invention further includes an Internet analysis system that includes a data set initializer to initialize a data set, a search engine to acquire search results, a bot to parse the search results, rather than a native search engine indexable resource, to output a conclusion, and an updated date set.

FIELD OF THE INVENTION

The present invention relates to the field of dynamic database searchingand more specifically to the field of electronic data analysis.

BACKGROUND

Internet search engines store information from a vast array of web pagesretrieved from the Internet, typically implemented through the use ofspiders or crawlers. To facilitate the search process, the Internetsearch engines provide interfaces used to run queries against theindices they build from this information. Generally speaking, Internetsearch engines build these indices by collecting URLs and following eachURL on each page until all URLs for all web pages have been exhausted.During this process, the contents of each web page are analyzedaccording to various and evolving criteria to determine how particularelements (e.g., titles, headings, files, links, various meta data, andthe like), and other related information should be indexed. This indexallows information to be found quickly, relevantly, and broadly from asingle source.

The automated collection of data available on the Internet is acomplicated task. According to U.S. Pat. No. 7,647,351 there isrecognized only one primary known means of automatically retrievinginformation from a web site (without the assistance of the web siteowner) utilizing the hidden mark-up language of the web site forcorrelating useful data. Theoretically, this mark-up can help a computeralgorithm locate, process, and interpret information on and about apage. As further noted by the '351 patent, “unfortunately, every Website has a different look and feel, so each Web page needs its owncustom algorithm. Writing a custom algorithm is time-intensive, butpossible on a small scale, such as a price comparison website whichgathers product information from a dozen sources. But there is noefficient way to scale this approach up to thousands or millions of Websites, which would require thousands or millions of custom algorithms tobe written.” The '351 patent attempts to solve data conformity problemsby the use of a manually set up template for each web page with a uniquelook and feel.

In fact, the computer system seeking to process resources (e.g., webpages, news feeds, PDF documents) available on the Internet is facedwith an earlier problem: locating those resources in the first place. Insome circumstances, the particular Internet locations of the resourcesto be processed and interpreted are known a priori (i.e., this resourceand that resource, located at these URLs) and can be accessedaccordingly. In others, no such knowledge exists, except in the abstract(i.e., it is suspected that the information is available somewhere, butit is not known specifically where).

Therefore, there is a need for flexible Internet data search processthat can meaningfully analyze and interpret data from disparate Internetresources, without accessing those resources directly, and withoutforeknowledge of the existence of or locations of such resources.

SUMMARY

The present invention is directed to an Internet analysis system andprocess for performing analysis and drawing conclusions based on dataacquired from the Internet. The system of the present invention includesan initial data set with input data, a non-party search engine, a searchinitializer, a bot, and an updated data set. The initial data setincludes investigation data consisting of a set of entities (e.g.,persons) identified by their key attributes (e.g., address) andoptionally described by zero or more additional attributes (e.g.,business ownership, picture) which can be populated or void. This datais to be tested with respect to a specific investigation activity,occurrence, or other criteria. The populated attributes contain dataprior to the analysis and may be updated (or voided) as a result of theanalysis; void attributes lack data, i.e., are void, but may becomepopulated later as a result of the analysis.

The nonparty Internet search engine of the present invention includes asearch engine that is not affiliated with either the party searching forinvestigation data on the Internet or websites on the Internet that maycontain the investigation data, or indicia thereof. The searchinitializer is loaded with search keyword data related to the populatedattributes and other words and phrases related to the transaction thatis the subject of the investigation. The search initializer thenaccesses the nonparty search engine and executes a search to generatesearch results, with search result entries and their search result entrydata, that is queued and/or paginated by the search engine according toits particular policies. The bot, having been manually loaded withevaluation criteria, parses the search result entry data to acquire datafor its analysis and potential subsequent update of one or moreattributes, entities, or other data species of the initial data set. Theinitial data set is thus transformed into the updated data set which mayinclude data related to the initial data set as well as the initial dataas supplemented, modified, or culled, by the bot.

These aspects of the invention are not meant to be exclusive.Furthermore, some features may apply to certain versions of theinvention, but not others. Other features, aspects, and advantages ofthe present invention will be readily apparent to those of ordinaryskill in the art when read in conjunction with the followingdescription, and accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a view of the system of the present invention.

FIG. 2 is a view of the system of the present invention.

FIG. 3 is a view of the system of the present invention.

FIG. 4 is a view of the process of the present invention.

FIG. 5 is a view of the process of the present invention.

FIG. 6 is a view of the process of the present invention.

FIG. 7 is a view of a web page of the present invention.

FIG. 8 is a view of a search result of the present invention.

FIG. 9 is a view of the system of the present invention.

FIG. 10 is a view of the process of the present invention.

DETAILED DESCRIPTION

Referring first to FIGS. 1 and 5, the present invention includes anInternet analysis system 100 and Internet analysis process 200. Theinvention operates to utilize a data set to perform an investigation,and preferably, alter a data set based on an analysis of Internet data.The data may comprise any data type and the present invention is notrestricted to any particular purpose or datum unless otherwise,expressly noted. In the preferred invention, the data can be describedas a series of entities that may the subject of a test. The entities arecharacterized by a series of attributes related to the entity. Preferredembodiments of the invention may be utilized as an investigation tool.In an investigation embodiment, the entity data may include a series ofindividuals such as property owners. Attributes for an individual may,by way of example, include inherent data for the person such as height,weight, age, address, etc. The data set of the present inventionincludes input data which is the data present in the system or processprior to a search and captured data, which includes the data acquiredfrom a search and either added to the data set or utilized to otherwisealter the data set.

The present invention determines through an investigation the likelihoodof an attribute, activity, or action (“conclusion”) applying to theentity based on an analysis of the results of a search or severalsearches. The conclusion need not be accurate or objectively true,conclusion for the purposes of the present invention is simply an answeras drawn by the present invention based on its analysis. A preferredembodiment of the analysis system 100 includes an initial data set 102,a search initializer 104, a non-party Internet search engine 106, a bot108, and an updated data set 110. By a non-party Internet search engine,it is meant a search entity that is not affiliated with either the partyperforming the analysis or the party holding the data that may berepresented in search results presented by the nonparty search engine.

The database of the present invention is expressed in an initial dataset 102. The initial data set 102 includes tabulated attributes 130 ofany nature. While the present invention utilizes PostgreSQL for its datastore, other SQL or non-SQL data storage and management systems couldalso be used. The data sets 102 can be seen to include 2 dimensions ofinformation. The entities (e.g., persons) are dispersed in the y-axis(represented in FIG. 1 by letters: a, b, y), and information datarelated to the same entity (e.g., characteristics), are dispersed in thex-axis (represented in FIG. 1 by numbers: 1, 2, X). There is no limit toeither the quantity of entities or attributes that may occupy the datasets of the present invention. Lastly, though the information isrepresented in two dimensions here for the purposes of explanation, thisis not a limitation of the present invention.

Turning to FIG. 3, the preferred initial data set 102 of the presentinvention is depicted. The initial data set 102 includes as entityattributes 130 identifying characteristics. Preferred investigation dataof the present invention includes property owner data, the identifyingcharacteristics of which data constitute the attributes 130. Theproperty owner data is expressed as entities along the y-axis asdifferent property owners while the attributes expressed along thex-axis include the identifying characteristics of the different propertyowners. Preferred identifying attributes include a name, a telephonenumber, email address, and any other characteristic that can be relatedto a particular property owner.

Attributes may be populated or void. Populated attributes have apre-existing value, irrespective of accuracy. Void attributes lack avalue. An example of a void attribute for property owner data mayinclude a null value as a telephone number. If the present inventiondetermines a telephone number for the investigation data, here propertyowner data, then the present invention may assign a value to thetelephone number attribute (in the case of a void attribute), or updatethe value to the most recent one found (in the case of a populatedattribute). The present invention implements this functionality byutilizing normalized tables within its data store, with telephonenumbers being stored in a separate table, and joined or associated withaddresses and people via a join table. The void attribute data may belater filled with a value—or not—depending on the uses of the presentinvention.

The search initializer 104 of the present invention is an access vehiclefor communicating with and providing instructions to a non-partyInternet search engine. The instructions provided to the non-partyInternet search engine may be in the form of keyword data to provide tothe search engine. In a preferred tax compliance system depicted in FIG.3, the system 100 includes an embodiment of the search initializer 104with transaction or conclusion keyword data and attributes, or words andphrases related thereto, for the entity under investigation. Conclusionkeyword, also known as transaction keyword, data includes keywordsrelated to the transaction with taxation consequences. The transactionkeyword data can include primary and secondary transaction keywords. Aprimary transaction keyword is a keyword set that directly relates tothe transaction/conclusion of consequence; for example, in a real estatetransaction/conclusion, the transaction keywords may include a name, atelephone number, an e-mail address, a physical address (e.g., city,state, country, etc.), a price, target consumer characteristics, realtycharacteristics (e.g., rooms, amenities, location, relative location,etc.), and any characteristics of a real estate transaction. Secondarykeyword includes keywords that are topically once or further removedfrom words and phrases that directly describe a transaction. Examples ofsecondary keywords in a real estate transaction include synonyms andlesser-used equivalents of primary keywords, transaction histories for aparticular vendor, and the like. The search initializer 104 reaches outto access the Internet search engine 106.

The search initializer of the present invention has been designed insuch a way that adapters are created for each search engine that is tobe used (e.g., Bing, Yahoo, Google). This allows common initializationfunctionality to be shared across adapters, and specific non-partyInternet search engine integration is handled within the associatedadapter. Each non-party Internet search engine adapter may vary tohandle things like differing search syntax and application programminginterfaces (APIs). For example, Bing assumes that searches with morethan 5 keywords can be “relaxed” for (i.e., not necessarily require thepresence of) the sixth and subsequent keywords. For the searchinitializer, the presence of every word within the search is desired;therefore the adapter for the Bing search engine adds the search enginespecific syntax of “norelax:” when using a long list of keywords.

The non-party Internet search engine 106 of the present invention isdesigned to aggregate and index information accessible on the World WideWeb, FTP servers, and other information collections accessible tocomputers. All information that is subject to indexing on an Internetsearch engine is known herein as search engine indexable Internetresources. The information such as it is possessed by the Internetsearch engine is known herein as search engine content. The informationsuch as it exists from an original, non search engine source, is knownherein as a native search engine indexable Internet resource, e.g.native web sites. The most common form of a search engine indexableInternet resource is a web site. The output of the search is generallypresented in a listing of search results 120 of search result entries116, via which the user may continue on to native or engine-hostedcached versions of the associated resources.

An Internet search engine is a preferred mechanism for accomplishing theobjectives of the present invention as it obviates the necessity of thesearching party to discover, a priori, the locations of the desiredresources, invent an original search algorithm, purchase substantialprocessing equipment, and duplicate storage and processing efforts.Search engines are easily and reliably accessible, supported by sizablecomputer processing power, and provided to the Internet communityspecifically for the purpose at issue, i.e., provide search results inresponse to specific queries taken from a large and growingcross-section of the Internet. The use of the Internet search enginealso avoids a number of problems that may arise when accessing nativeweb sites and resources, including excessive loads placed on web siteservers and bandwidth. It is an aspect of the present invention that theassistance of an Internet search engine is relied upon, not only toprovide substantial processing power, but also to provide generalsorting algorithms. Because search engines are specialized for fastsearching that results in useful sorting, the present inventionincorporates the methodology reflected in the results provided by any ofa number of established Internet search engine providers. A drawback isthat the algorithms used by a particular search engine may not bepublicly ascertainable, but notwithstanding that secrecy, search enginescan generally be relied upon to furnish search results in a fashion thatis broadly applicable to various search functions. The search enginesare taken “as is” and their return of data in a particular order isadopted by the present invention, as is their discretion in returningmultiple or no results for a particular search. The present inventioncan incorporate any combination of search engines to support itsanalysis and counteract any problems with any specific engine'sparticular algorithms.

The search initializer 104 acts to initiate a search that yields a setof search results 120 with search result entries 116 from the nonpartysearch engine. The bot 108 of the present invention, rather thancontinuing on to access the native website 112 from the search resultentries 116, as is general practice, analyzes only the search results120 provided by the search engine 106 in response to the search requestby the search initializer 104 and wholly bypasses and ignores the nativeweb sites 112. As many web sites expressly invite search engines toharvest web site content for purposes of search engine optimization andthe like, the applicability of the present invention is very broad.

Returning to FIG. 1, the bot 108 is loaded with evaluation criteria. Theevaluation criteria preferably include keywords, phrases, and othermeasurement criteria in addition to attribute data, but related to theactivity being analyzed/investigated. The keywords of the evaluationcriteria and those utilized by the search initializer differ in that thekeywords of the search initializer are adapted to generate searchresults (and are usually of lesser quantity), while the keywords, etal., of the evaluation criteria include a significantly broader range ofkeywords and evaluation data. In a database augmentation embodiment ofthe present invention (described above with the present inventionutilizing PostgreSQL), the bot 108 uses information from the searchresults 120 to supply missing or desired data to the initial data setresulting in an updated data set 110. In any embodiment of the presentinvention, the updated data set 110 can either be a modified version ofthe initial data set 102, resulting in only a single data set, or can bea distinct data set different from the initial data set 102, resultingin two or more data sets. In the database augmentation embodiment of thepresent invention, the bot 108 proceeds through as many of the entitiesas is desired and supplies information for the updated data settransforming void attributes to populated attributes as desired. In thepresent invention the transformation of attribute data is achieved byeither updating an existing database record with the new data, orcreating a new record that is then associated with previously existingdata via foreign keys within a join table. The information retrieved bythe bot is unrestricted in its nature. The keywords of the bot should bechosen to assist the bot in accurately interpreting the search resulttext 116 to retrieve accurate information for the void attribute data orother function performed by the bot according to the present invention.

The bot 108 may parse the search result entry data 134 of the searchresults 120 pursuant to any means known in the art. A first preferredmeans of ensuring accurate parsing of search results as evaluationcriteria involves the uses of structured data. Structured data isrecognizable among unrelated data, as structured data possesses anidentity distinct from the information that it represents. That is tosay, a telephone number can be recognized as a telephone number on thebasis of its numbers (its information), its depiction as the number ofdigits that the telephone number possesses (nine for standard telephonenumbers), or its format, such as a particular hyphenation or spacingpattern associated with telephone numbers (e.g., ###-###-####,(###)###.####). Structured data may take many forms, including the meansof portraying a name, address, location, email address, a URL, etc.

The bot 108 may parse information present in search results 120,particularly search result entry data 134, according to loadedevaluation criteria, provided as initial or updated instructions as tothe relevance of the information subject to being passed to the updateddata set 110. Search result entry data includes the information, suchtext, media, graphics, files, etc. that may be available directly fromthe search engine results without further recourse to the search engineindexable Internet resource. It is preferred that the evaluationcriteria be manually loaded such that the criteria is supervised byhuman handlers. For example, the bot may be instructed to compare URLresults with a URL database; the result originating from a particularURL in that database may be an indication of validity or invalidity. Inpractice, the bot 108 may be allowed to proceed with the parsing of thesearch results and paused after a significant period of activity, thatis say, the bot need not parse every search result entry for the searchentry data therein. The results of the updated data set may be reviewedmanually for the relevance and accuracy of the information passed fromthe bot thereto, i.e., information that has been used to transformedvoid attributes to populated attributes, and the results of such reviewcan be used to update the instructions used by the bot used asevaluation criteria in subsequent operations. Valid captured data, i.e.,data that matches the requirements of the data being sought, can beaggregated to ascertain similar characteristics of the valid data. Anexample of frequently valid similar characteristics includes a commonURL source, particularly the tax compliance embodiment of the presentinvention. Information can be determined as having greater validity ifit originates from a URL source supporting and encouraging suchtransactions with taxation consequences, e.g., craigslist.org, forrental offers.

The factors utilized as evaluation criteria are without limit and may beany factor related to an activity that is the subject of the analysis.Exemplary evaluation criteria include: whether there are search resultsat all; whether the search results contains attributes, or relatedinformation, or keywords at all; a determination of the informationcontent in the search results; the number of search result entriesreturned; the frequency of the attributes, or related information, orkeywords in the search results; a determination of the information ofthe meta tags; the domains from which the results originate; adetermination of the information extracted from the domain which theresults originate; etc. Each of the above criteria is considered searchresult data.

A whitelist of URLs can be aggregated as evaluation criteria in the bot.Information originating from a whitelist URL can be immediately, orgraded as more likely to be, passed to the updated data set. Converselya blacklist of URLs can be aggregated as evaluation criteria in the botto grade web sites as less likely to be passed to the updated data set.As an example for the taxation compliance embodiment, a whitelist mightinclude a craiglist.org for rental offers while the blacklist mightinclude informational do-it-yourself web sites related to constructionprojects, or reverse phone number web sites. Information from the bot'sparsing deemed initially to have potential to be passed to the updateddata set might be negated and the present invention may re-parse thesearch results for a second or other candidate, move to the nextdistinct entity of the initial data set without attempting to provideinformation to void attributes, or perform another operation of thepresent invention. Rather than analyze the content of the searchresults, the bot may be loaded solely with URL comparison data, suchthat the existence of any search result entry corresponding to awhitelist URL equates to the validity of the activity subject toanalysis. Another operation of the present invention includes thefiltration embodiment of the present invention depicted in FIG. 2.

Turning to FIG. 2, the filtration embodiment of the system 100 of thepresent invention includes a Boolean filter 134. A primary differencebetween the filtration embodiment and the augmentation embodiment isthat the filtration embodiment acts to reduce the initial data set 102to an updated data set 110 that includes a diminished quantity ofentities relative to the initial data set entities. The filtrationembodiment may be used in operations when a user desires not to expand adatabase of known information constituents, but rather to determinewhether entities have or have not engaged in a particular activity. Theinitial data set 102 passes entities with their related attributes tothe search initializer 104, which accesses and sends instructions to asearch engine 106, which uses its repository 122 of stored websites andits particular search algorithms to produce search results 120. The bot108 parses the search results 120 and a Boolean filter 134 that includesinstructions therein, or communicates with another component of thepresent invention bearing instructions, to determine whether the entityhas been confirmed as engaging in an activity on the Internet. If theentity is confirmed as engaging in an activity the Boolean filterreturns a value as “true” and either retains or deletes the entity inthe updated data set 110 or provides an updated data set that includessome indicia of the entities categorized as either “true” or “false.” Ifthe entity is not confirmed as engaging in an activity the Booleanfilter returns a value as “false” and either retains or deletes theinvestigation data in the updated data set 110.

An example of a preferred use of the filtration embodiment of thepresent invention is to search for entities that engage in an activity.As an example, the entity data may be composed of property owners withina community. Property owners that engage the activity in question, areparsed via the bot 108, and then rather than augment the entity datafrom the initial data set 102 the Boolean filter 134 deletes “false”entity data to create an updated data set 110 consisting solely of“true” entity data. If the activity includes offering the property forrent, then the updated data set 110 includes only those property ownersas entities suspected by the present invention as engaging in offeringthe property for rent. Alternatively, the filtration embodiment of thepresent invention can be functionally replicated by the augmentationembodiment by utilizing a Boolean as an entity attribute in the initialdata set 102 that is modified by the bot 108.

Turning now to FIG. 3, the present invention may also generate anupdated data set 110 that includes captured data 136. An aspect of thepresent invention includes the scouring of the Internet to acquirecaptured data based on the input data. The captured data may include newattributes of pre-existing entities of pre-existing categories (in afashion similar to that of updating void data attributes to populatedattributes), new entities that may include one or more attributescorresponding to attributes pre-existing or otherwise, new attributes ofexisting entities in categories pre-existing or other, and the like. Itis preferred that captured data correspond to the existing structure ofthe initial data set; that is to say, the entry of a new entity or applyto a pre-existing entity attribute category even if occupied by voidplaceholders. However, the present invention may act as a scouring agentthat acquires data related to the activity that is the subject of theanalysis and add such data as information as an attribute for an entityirrespective of applicability to a category. Exemplary captured data mayinclude narratives about/from the entity, media about/from the entity,data files, etc. Captured data acquired by the bot 108, or a link toexternal data file as a proxy for the captured data, may be positionedin the updated data set 110. Another preferred form of captured data 136includes a link to the Internet source serving as the basis formodifying or updating the data in the updated data. As the presentinvention does not visit a native, contemporaneous web site, it ispreferred, as in shown in FIG. 4, the means by which a reviewing partymay audit the results of the data set manipulation is to associate 210 alink with the entity data, to permit a third-party reviewer a means toaccess the native web site, or cached version thereof, containing thedata upon which the conclusion was drawn. Furthermore, any data asexisting in the updated data set as different from the data in theinitial data set, whether such differences derive from transition ofvoid attributes to populated attributes, the acquisition of captureddata, the addition of entities, etc., may be reconstituted 250. Byreconstituted, it is meant that the data of the updated data set isarranged as initial data for one or more further passes through othersteps or components of the present invention. By re-cycling modifieddata through the steps and components of the invention, the accuracy ofthe invention is enhanced because outdated, inaccurate data is discardedfor new, accurate data and pre-existing data may be supplemented withadditional data.

Returning to FIG. 3, an example of the present invention utilizingcaptured data is a job-posting embodiment of the present invention. Theinvestigation data may include employment opportunities with the hiringbusiness as the entity and the business name, business address, andbusiness telephone number as entity attributes data and the position andsalary as void attributes. The search initializer 104 may access anon-party search engine 106 to generate search results 120 from which abot 108 parses search result entry data 134. Upon determining the salaryand position of specific business entities, the bot may further capturethe narrative related to the position or the narrative characterizingthe rationale related to a salary range. One means for determining theexistence of, for example, the narrative as captured data is to supplythe bot 108 with keywords relating to a position description as a firstevaluation criterion and then grammar syntax cues relating to thebeginning and end of a suspect narrative (e.g., paragraph indentations,seek root word “requirement” in first sentence, seek bullet marks, etc.)as the second evaluation criterion.

In a Request-for-Proposal embodiment, the present invention may capturethe request for proposal document as a standalone file as captured data.The investigation data may include agency projects with the agencyproject as the entity data and the agency name, agency address, andagency telephone number as entity attributes and the project name asvoid related data. Upon determining the project name, the bot mayfurther capture the RFP document for persistent retention. To ensurethat the file capture does not involve accessing the native website, thebot may be loaded with instructions to acquire such files merely fromthe non-party search engine, or upon finding that the non-party searchengine lacks a required file, captures link data describing the locationof the file. Alternatively the captured data, particularly when files,may be captured directly from the native website.

The captured data may further be used in the evaluation criteria of thebot. The bot may be manually loaded with keywords and informationrelated to positive correlations, i.e., whitelist, or negativecorrelations, i.e., blacklist. Positive correlations with keywordsinclude keywords that make the existence of a fact more likely than not,and the negative correlations with keywords include keywords that makethe existence of a fact less likely than otherwise. Use of keywordcaptured data with the evaluation criteria is particularly useful whentransaction keywords in the search initializer are likely to generatesearch results that include unrelated search result text snippets andunrelated historic web site data. For example, in the RFP embodiment ofthe present invention, the evaluation criteria may include as whitelistcomponents verbs related to the provision of a request for proposal,such as the keywords: “contractor,” “bid,” keywords related tocredentials, and the like. The evaluation criteria may include asblacklist components verbs unrelated to the provision of a request forproposal but related to government agencies. For example, if requestsfor proposals frequently share general terms with the frequently askedquestions pages of government agencies, then appropriate blacklistcomponents may include such keywords as: “FOIA,” “Privacy Act,” “RequestRecords,” and the like.

The confidence, or accuracy, detector 132 of the present inventionprovides an attempt to measure the confidence of the intended updatesthat would form the updated data set 110 if applied. The confidencedetector differs from the evaluation criteria in that the evaluationcriteria attempts to qualitatively review a data candidate purely forthe purposes of inclusion or exclusion. A preferred version of theconfidence detector includes a pre-defined scoring system attributing aspecified score to the findings of a search in an attempt toquantitatively measure the results of the search. Another generalmethodology for scoring the results of bot output or findings includesthe comparison of the bot outputs for different search engines accessedduring the searching step of the present invention. The process of thepresent invention may run in a parallel series, such that a first use ofthe process searches GOOGLE, a second use of the process searches BING,a third use of the process searches YAHOO, etc. Information may be foundin the differences between the outputs of the bots for each differentsearch engine. Generally, scores may be positive or negative to producea positive or negative score total, or any other basis for scoring orgrading confidence may be utilized.

Turning to FIG. 6 in view of FIG. 1, a simplistic, fictional example ofa present invention is presented in which there is an investigation inan attempt to determine literary characters that have visitedWonderland. The initial data set 102 is initialized 202 to include, asentities, literary characters: the attributes may include the charactername, character physical characteristics, and character location aspopulated attributes, and a Boolean value representing whether it istrue that the character visited Wonderland as a void attribute. Thesearch initializer 104 is loaded with the populated attributes of theentity and transaction keyword data and accesses 204 a search enginerepository 122. The initial data set 102 may include such entities andattributes as represented in Table 1.

TABLE 1 Examples of Entities and Attributes in Initial Data Set Alicegirl who wears a Britain Var_Visit pinafore Wonderland? Deerslayer manin leather Forests of North Var_Visit chaps U.S. Wonderland? Babe blueox U.S. Var_Visit Wonderland?The search initializer might include such search strings asAtt_(—)1+Att_(—)2+Att_(—)3+“Wonderland”+((“fell” or“fall”)+“well”)+“mirror.” The Att_X search terms include reproductionsof the populated attributes and the remaining terms include transactionkeywords relating to investigation of the activity of visitingWonderland, which for the sake of the present disclosure is onlyaccessible through wells and mirrors. A search term as passed to thesearch initializer may include, in the case of the first distinct entityof the investigation data: “Alice”+“girl who wears apinafore”+“Britain”+“Wonderland”+((“fell” or “fall”)+“well”)+“mirror.”

In the embodiment of FIG. 6 with reference to FIG. 2, the searchinitializer 104 accesses 204 the search engine to retrieve the searchresults 120. An example of a web site data 114 referred to by searchresult entry 116 may resemble that of FIG. 7. The present invention actsto approximate the results of a thorough analysis of native web pagecontent, such as the web site data 114 of FIG. 7, without ever accessingnative web pages 112 or the native web page data 114 with the efficiencyof relying solely on an Internet search engine. The search enginerepository 122 may include an identical cached version of the native webpage 112 as content 124, a previous version, a similar version, or notinclude a version of the native web page at all. Another aspect of thepresent invention is the ability to analyze data from the Internet, asit existed historically; when such an objective is sought, it may beadvantageous to include a time stamp as a captured data attribute in theupdated data set.

In continuing the ‘Wonderland Example’, and as shown by FIGS. 1, 6 and8, the search initializer 104 may access 204 the search enginerepository 122 to receive, as a search result 120 for Alice as an entityin the initial data set 102, the search results 120, search resultentries 116, and search result data data 134 of FIG. 8. As can be seenby FIG. 8, rather than reproduce directly or indirectly web site text,the present invention parses and analyzes 206 only the search results120, specifically and preferably the search result entry data 134, andthe search result entries 116 as truncated, ordered, and otherwiseedited, by the search engine 106 by using the bot 108 to test theinvestigation data. If the bot 108 is prepared 260 with evaluationcriteria that includes such transaction keyword search terms related tovisits, prominent Wonderland citizens (e.g., the Rabbit), etc., then thebot 108 can determine as an output conclusion that Alice is valid dataand provide 208 an update to the updated data set 110 accordingly. Thebot 108 in reviewing the search results 120 may evaluate each searchentry individually, in groups, or in their totality to test theinvestigation data, entity by entity. Furthermore, the bot 108 mayreturn an action related to investigation data as valid merely by theexistence of any, or a particular quantity of, search results for thesearch initiated by the search initializer.

In continuing the ‘Wonderland Example’, and as shown by FIGS. 1, 6 and8, the search initializer 104 may access 204 the search enginerepository 122 to receive, as a search result 120 for Alice as an entityin the initial data set 102, the search results 120, search entities116, and search entity data 134 of FIG. 8. As can be seen by FIG. 8,rather than reproduce directly or indirectly web site text, the presentinvention parses and analyzes 206 only the search results 120,specifically and preferably the search result data 134, and the searchresults entries 116 as truncated, ordered, and otherwise edited, by thesearch engine 106 by using the bot 108 to test the investigation data.If the bot 108 is prepared 260 with evaluation criteria that includessuch transaction keyword search terms related to visits, prominentWonderland citizens (e.g., the Rabbit), etc., then the bot 108 candetermine as an output conclusion that Alice is valid data and provide208 an update to the updated data set 110 accordingly. The bot 108 inreviewing the search results 120 may evaluate each search entryindividually, in groups, or in their totality to test the investigationdata, entity by entity. Furthermore, the bot 108 may return an actionrelated to investigation data as valid merely by the existence of any,or a particular quantity of, search results for the search initiated bythe search initializer.

To further safeguard accuracy, as can be seen in FIGS. 1 and 6, thepresent invention may utilize enhanced evaluation criteria and aconfidence detector 132. Enhanced evaluation criteria may includepositive filtration 220 and negative filtration 222, utilizing thepositive filters (i.e., whitelists) and negative filters (i.e.,blacklists), respectively, discussed herein. The confidence detector 132constitutes an attempt to mathematically score data manipulated withinthe updated data set 110 or the differences between the initial data set102 and the updated data set 110. Scores may be provided for particularpositive words, phrases, locations, and the like, and scores may beprovided for particular negative words, phrases, locations, and thelike. The output of the confidence detector for each distinct entity maybe inserted into the updated data set 110 or otherwise associated withthe distinct entity.

Turning now to FIGS. 9 and 10, a preferred embodiment of the taxcompliance, i.e., VRCompliance, system 100 and process 200 and of thepresent invention is used to monitor tax compliance by vacation rentalsby property owners. An increasing number of second-home owners arerenting their homes as vacation properties. Using online advertisingmeans, many of these property owners are able to market their propertiesand handle bookings at a minimal cost without using a propertymanagement firm. In many such cases the owners are unaware that rentinga property often requires compliance with state and community specificsales/lodging tax and business licensing requirements. There is apotentially large and increasing amount of community revenue that islost due to the non-compliance of the Vacation Rental By Owner (VRBO)property owners. Additionally, there are significant market distortionsthat arise from varying degrees of compliance within a particularmarket.

The growth of the VRBO market will continue to drive the complianceproblems experienced by resort communities and revenue and taxdepartments. Currently, determining compliance can be a labor-intensiveprocess because there are a number of websites where individualsadvertise their properties, and a number of additional disparate andnon-correlated data sources that in the aggregate comprise the completedata set required to assess compliance. The properties are generallyonly listed with a phone number or an email address for contacting theowner. There is a lack of critical information such as the full name ofthe owner and the address of the vacation home, including unit number.Currently a community 190 is left with the decision of whether to makedirect contact by calling the owner or search for the owner's propertyaddress via indirect means. Some communities 190 search for theproperties based on photos posted on the website. Regardless of howcommunities try to identify these properties and their owners, theprocess continues as listings are updated, added, and deleted daily fromonline sites.

The VRCompliance methodology simplifies the process of matching vacationrental property advertisements to property owners, and verifying whetherthe property owners are in compliance with tax and licensingrequirements for the community. The methodology is as follows: (a)Import Property Tax Records For A Community; (b) Import ComplianceRecords For A Community; (c) Import Additional Data On Owners FromExternal Data Sources; (d) Create Queries Using Gathered InformationThat Should Result In Advertisements; (e) Process Queries Using SearchEngines; (f) Filter Query Results To Show Information That Is MoreLikely To Be An Advertisement (g) Provide The User The Results In ADigestible Format.

Property tax records 196 for the community 190 are obtained for thecommunity and imported into the VRCompliance system as populatedconstituent data 130. Because each community 190 has their tax recordsin a different format a normalization filter 180 is required. Apreferred normalization filter 180 utilizes a Domain Specific Language(DSL) may be employed to aid in importing records from various formatsinto the common format that is used by the VRCompliance system. A DSL isa programming language with syntax dedicated to a particular problemdomain, a particular problem representation technique, and/or aparticular solution technique. The preferred DSL for property importingincludes the following language constructs:

-   -   community_name(name)—The community name can be specified. If it        already exists in the VRCompliance system, then that community        object is used; otherwise a new community object is created and        saved within the VRCompliance system. map(vr_column_name,        input_column_name, options={ }, &block)—The map construct allows        one to declare a mapping between an input column from the        community data, and our output column or column names. The map        construct can apply output filters to run prior to assigning        data to the result, or it can take a block for more complicated        processing. See the following examples:

Example 1

The following is a simple example that maps the value in column 39 ofthe input file to the VR_ADDRESS2 column in the output map“VR_ADDRESS2”, 39

Example 2

The following example shows that column 3 is should be run through anormalizing and capitalization routine prior to being assigned to theoutput column VR_MAILING_ADDRESS_CARE_OF.

map “VR_MAILING_ADDRESS_CARE_OF”, 3, :methods=>[:normalize upcase]

Example 3

The following example is a more complicated manipulation of multiplecolumns of incoming data that is then assigned to a single row in theoutput. In this case the VRCompliance address 1 is comprised of columns34 through 38 of the incoming data. The address is normalized and madeto be uppercase. In this case the incoming data sometimes has a value of“NO ASSIGNED ADDRESS”. However, the internal representation for that is“N/A”.

 map “VR_ADDRESS1”, :na do |row|    address_num = “”    begin    address_num = Float(row[34]).to_i.to_s    rescue Exception => e    address_num = row[34]    end    address = [address_num, row[35],row[36], row[37], row[38]].compact.join(“ ”)    address =address.normalize_upcase.gsub(/{circumflex over ( )}0+/,″) unlessaddress.nil?    if address == “NO ASSIGNED ADDRESS”     address = “N/A”   end    address   endThree additional constructs of the DSL allow assigning of regularexpressions that can be used to determine if the listed owner is abusiness, a trust, or a person or persons.

business_words(regular expression)

trust_words(regular expression)

person_words(regular expression)

Additionally there is a construct for parsing owner names.

-   -   owner_parser(&block)—The owner parser accepts a block of custom        code that is used to parse the normalized owner data. The        resulting data will be one or more person objects in a form that        is usable by the VRCompliance system.        The final construct is the call that specifies the input file        name that should be loaded.    -   import_properties (file_name, options={ })—The file_name is a        file on the file system that should be imported, and the options        describe the file such as whether the file has a header row.

By setting up a configuration block using this internal language processcommunity tax records can be more simply formatted in a style convenientto the VRCompliance system.

Importing compliance records 196 of a particular type for a community aspopulated constituent data 130, in the VRCompliance methodology, may beachieved using the DSL techniques described above. The VRCompliancemethodology may be applied to any compliance type, e.g. a businesslicense or a sales and lodging tax filing. Communities provideinformation on property owners within that community that are compliantin these contexts. Once imported, within the VRCompliance methodology anassociation is created between the property and its associatedcompliance records.

The methodology includes importing additional data on owners, such asphone number and email addresses, from external data sources 198 tocreate additional populated constituent data. Thus this embodimentutilizes information wholly originating from the community 190 andnon-Internet sources 198 to populate 202 the initial data set 102 of thepresent invention.

The methodology then populates the search initializer 104 usingconstituent data and other information as keywords calculated to yieldresults that include relevant advertisements. For each property owner(the entity) in a community a custom query is created 204 to search forpotential advertisements. These custom queries are tested 240 foruniqueness and then sent to the search engines 106 via a customVRCompliance interface 242 in the search initializer 104. TheVRCompliance system then iterates through the custom search results 120for each user. The custom searches utilize unique information that isassociated with the owner such as their phone number or email. Based onthe uniquely identifiable information, the results of these searches aretied to the person and the rental property(ies) owned by that person.

The methodology filters query results to consider information that ismore likely to be an advertisement. Simply because a search result isfound does not mean that it is an advertisement for a rental property.In order to minimize the amount of noise received in search results theVRCompliance methodology parses 206 the results via a bot 108. Searchresults are filtered based on whether the domain name that the searchresult points to is on a whitelist 220 or blacklist 222 that has beenanalyzed by human analysis or can be ignored. The results are marked 280and passed to a Boolean filter 134. A return 282 continues the process200 until the initial data set is exhausted of distinct data. Onlysearch results that are associated with domains on the white list 220are displayed for a given property in a community and are recognized assuch because the updated data set includes as constituent data 130 aBoolean validating/negating the possibility of tax compliance. Thesesearch results are considered advertisements and are associated with theproperty in the community that is owned by the property owner. Uponfinishing 286 the process, the updated data set may be viewed.

Although the present invention has been described in considerable detailwith reference to certain preferred versions thereof, other versionswould be readily apparent to those of ordinary skill in the art.Therefore, the spirit and scope of the appended claims should not belimited to the description of the preferred versions contained herein.

What is claimed is:
 1. An Internet analysis system, said system comprising: an initial data set of input data including a series of entities, each of said entities characterized by a series of entity attributes; a nonparty Internet search engine with a repository of stored content from search engine indexable Internet resources; a search initializer, loaded with search keyword data derived from said entity attributes, adapted to access said search engine to generate, for at least one of said entities, a search result of search result entries, with search result entry data, queued according to policies of said search engine; and a bot, loaded with evaluation criteria, adapted to parse said search result entry data, bypass native versions of said search engine indexable resources corresponding to said search result entries, to informationally evaluate said search result entry data to determine as an output an investigation conclusion concerning at least one of said entities; and an updated data set derived from said initial data set as informed by said output of said bot.
 2. The system of claim 1 wherein said search keyword data includes keywords related to said conclusion.
 3. The system of claim 1 wherein said initial data set includes attributes corresponding to predetermined, uniform categories.
 4. The system of claim 3 wherein said updated data set includes as at least one of said entity attributes said output of said bot.
 5. The system of claim 3 wherein said bot captures information from said search result entry data as captured data.
 6. The system of claim 5 wherein said updated data set includes captured data.
 7. The system of claim 5 wherein said captured data includes a destination pointer comprising a data link identifying a search engine indexable Internet resource from said search result.
 8. The system of claim 5 wherein said updated data set includes said captured data as at least one of said entity attributes.
 9. The system of claim 8 wherein said updated data set includes captured data as an entity.
 10. The system of claim 1 further comprising a confidence detector adapted to evaluate said captured data to provide a confidence rating of said output of said bot.
 11. The system of claim 1 further comprising a filter, adapted to review said output of said bot, manually loaded with at least one positive filtration criterion to automatically affect said output of said bot based on existence of said positive filtration criterion in association with said entity.
 12. The system of claim 1 further comprising a filter, adapted to review said output of said bot, manually loaded with at least one negative filtration criterion to automatically affect said output of said bot based on existence of said negative filtration criterion in association with said entity.
 13. The system of claim 1 wherein said output consists of a Boolean value.
 14. The system of claim 13 wherein said updated data set consists of entities coupled with said entity attributes for which such output is true.
 15. The system of claim 13 wherein said updated data set consists of entities coupled with said entity attributes for which such output is false.
 16. An Internet analysis process comprising: initializing an initial data set of input data including a series of entities, each of said entities characterized by a series of entity attributes; accessing a nonparty Internet search engine with a repository of stored content from search engine indexable Internet resources; searching said search engine with search keyword data from said entity attributes to generate, for at least one of said entities, a search result of search result entries, with search result entry data, queued according to policies of said search engine; parsing said search result entry data with a bot, loaded with evaluation criteria, adapted to bypass native versions of said search engine indexable resources corresponding to said search result entries, to informationally evaluate said search result entry data to determine as an output an investigation conclusion concerning at least one of said entities; and providing an updated data set derived from said initial data set as informed by said output of said bot.
 17. The process of claim 16 wherein said providing step includes determining as said output a Boolean value.
 18. The process of claim 17 wherein said providing step includes providing said updated data set that consists of entities which such output is true.
 19. The process of claim 16 wherein said parsing step includes capturing information from said search result entry data as captured data and said providing step includes providing said updated data set with captured data as said entities.
 20. The process of claim 16 wherein said parsing step includes capturing information from said search result entry data as captured data and said providing step includes providing said updated data set with captured data as said attributes. 